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OPTIMIZING SCALEABLE VIDEO ALGORITHM ASSET DISTRIBUTION UTILIZING QUALITY 
INDICATORS 



In general, the invention relates to Scaleable Video Algorithms (SVAs). More 
specifically, the invention relates to a method and system for optimizing SVA asset 
distribution. 

10 Future consumer terminals such as, TV sets, set-top boxes (STBs), and displays 

combine high-quality video and audio with applications from the mainstream multimedia 
domain as found on personal computers (PCs). 

Future consumer terminals will be based on programmable platforms instead of - 
dedicated hardware. Execution of video algorithms on programmable platforms is limited 

15 by the available resources. Recently, scaleable algorithms and run-time control of output 
quality and resource usage have been utilized to overcome such limitations, such as, for 
example SVAs including MPEG-2 decoding and imaging enhancements and Quality-of- 
Service (QoS) control software. 

Scaleable algorithms can use programmable components in a cost-effective manner. 

20 An SVA is an algorithm that allows the dynamic adaptation of output quality versus 
resource usage on a given platform. Traditional systems do not support such dynamic 
control of the resources and change of quality levels of an algorithm. Software solutions 
must also result in stable, robust, predictable, and cost-effective systems. Therefore, a QoS 
environment must include dynamic resource management. 

25 An SVA supports different platforms/product families for media processing and is 

easily controllable by a control device for several predefmed settings. SVAs with kregular 
priority processmg start with the most important image parts and process data in decreasing 
order of importance. SVAs can be regulated or interrupted to meet resource limitations. 
Therefore, these SVAs are inherentiy data dependant with varying output quality. 

30 SVAs can be designed to allow for different quality levels in exchange for tiie 

required processing resources. In a dynamic enviromnent with scaleable algoritiims, the 
system optimization has to take both resources and quality into account. Lacking proper 
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information leads to suboptimal results. The lack of useful qualily information is a 
bottleneck in dynamic resource controlled systems. 

It would be desirable, therefore, to provide a method and system that would 
overcome these and other disadvantages. 

5 One aspect of the invention provides a method for controlling asset allocation of a 

consumer terminal by receiving input data into at least one scalable media algorithm, 
processing ttie input data through at least one scalable media algorithm, and determining at 
least one quality indicator value associated with each scalable media algorithm based on 
the processing for each scalable media algorithm. 

10 In accordance with another aspect of the invention, a computer readable medium 

storing a computer program includes: computer readable code for receiving input data into 
at least one scalable media algorithm; computer readable code for processing the input data 
through at least one scalable media algorithm; and computer readable code for determining 
at least one quality indicator value associated with each scalable media algorithm based on 

15 the processing for each scalable media algorithm. 

In accordance with yet another aspect of the invention, a system for controlling 
asset allocation of a consumer terminal is provided. The system includes means for 
receiving input data into at least one scalable media algorithm. The system further 
includes means for processing the input data through at least one scalable media algorithm. 

20 Means for determining at least one quality indicator value associated with each scalable 
media algorithm based on the processing for each scalable media algorithm is also 
provided. 

The foregoing and other features and advantages of the invention will become 
further apparent from the following detailed description of the presently preferred 
25 embodiment, read in conjunction with the accompanying drawings. The detailed 

description and drawings are merely illustrative of the invention rather than limiting, the 
scope of the invention being defined by the appended claims and equivalents thereof 

FIG. 1 is a block diagram illustrating an operating envirocment in accordance with 
the present invention; 

30 FIG. 2 is a block diagram illustrating a control system in accordance with the 

present invention; 
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FIG. 3 is a block diagram illustrating a scalable algorithm with quality indicator 
output in accordance with the present invention; and 

FIG. 4 is a flow diagram depicting an exemplary embodiment of code on a 
computer readable medium in accordance with the present invention. 
5 FIG. 1 is a block diagram illustrating an operating envirormient in accordance with 

the present invention. In FIG. 1, system 100 includes scaleable and non-scaleable 
algorithms that run concurrently on a programmable processor and coprocessors (not 
shown). Scalable algorithms include MPEG video decoder 130, sharpness enhancement 
135, and software Scaler 165. The software Scaler 165 provides down-scaling for picture- 

10 in-picture application. Non-scaleable algorithms include Demultiplexer 115, audio 

decoder 120, software mixer 140, hardware scaler 170, and MPEG encoder 175, Scalable 
algorithms are scalable media algorithms (SMA) that can be implemented as video (SVA), 
graphics (SGA), or audio (SAA) applications 180 

In operation, available system assets are distributed to the non-scaleable algorithms 

15 based on set requirements. Assets include resources such as CPU cycles, coprocessor 

cycles, memory, bus bandwidth, time, and the like. Typically, once the available assets are 
distributed to the non-scaleable algorithms then remaining assets are distributed to the 
scaleable algorithms. In one embodiment, the remaining assets are distributed to the 
scaleable algorithms based on the amoxmt of assets available and the number and type of 

20 scaleable algorithms that are operating. 

In an example, DVD imit 1 10 is the only device operating. After assets are 
allocated to Demultiplexer 115, audio decoder 120, and software mixer 140 the remaining 
assets are allocated to the two scaleable algorithms MPEG video decoder 130 and 
sharpness enhancement 135. 

25 Throughout the operation of system 100 additional assets, such as assets unused 

after initial allocation, may be available for allocating to scaleable algorithms. In one 
embodiment, additiorml assets are allocated in a predetermined manner, such as, for 
example based on an asset allocation table. 

In this example, when analog video unit 160 is introduced to the system, such as the 

30 use of an analog video imit 160 in conjunction with a picture-in-picture function 1 95 of 
display 190, additional non-scaleable algorithms will require asset allocation in addition to 
scaleable algorithm software Scaler 165. The increase in asset demand will require 
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reassessment of assets assigned to scaleable algorithms MPEG video decoder 130 and 
sharpness enhancement 135. 

FIG. 2 is a block diagram illustrating a control system 200 in accordance with the 
present invention. FIG. 2 includes scaleable video algorithm (SVA) 210, including priority 

5 processing, coupled to a system controller unit 220. SVAs including priority processing 
start processing with the most important image parts and process data in a decreasing order 
of importance. SVAs with priority processing can be regulated or interrupted to meet 
system allocation requirements. Therefore, SVAs with priority processing are inherently 
data dependant Additionally, the resulting output quality is usually not a function of the 

10 assets (resources) used for processing. 

Scaleable video algorithm (SVA) 210 further includes a quality indicator unit 230 
that is also coupled to the system controller unit 220. In one embodiment and referring to 
FIG. 1 above, SVA 210 is implemented as any of the SVAs, such as, for example MPEG 
video decoder 130, sharpness enhancement 135, or software Scaler 165. Quality indicator 

15 unit 230 is a software component that produces at least one quality indicator value based 
on the amount and type of processing SVA 210 completes. 

In operation, SVA 210 receives an asset allocation, also referred to as a budget 
from system controller 220 based on system requirements. SVA 210 also receives input 
data and processes the received input data into output data based on the amoimt of assets 

20 allocated. Quality indicator imit 230 analyzes the amount of processing, determines a class 
based on the analyzed amount of processing, and assigns the quality indicator value based 
on the determined class. 

In one embodiment, quality indicator imit 230 produces multiple quality indicator 
values based on different criteria. QuaUty indicator unit 230 transmits the quality indicator 

25 values to system controller 220. System controller 220 optimizes system assets based on 
the quality indicator values. 

Such a quality indicator is described for scalable motion estimation. Imagine, 
motion estimation entails establishing a block size including a specified grain detail for 
processing an entire frame. The process envisions starting with a large block size and 

30 course grain and processing the entire frame. In one embodiment, if enougih processing is 
available the block size can be decreased and the grain detail accuracy increased to 
determine a processing level. 
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A quality indicator value is detexmined by analyzing the smallest block size and 
finest grain processed. Ending the processing with a large block size or course grain detail 
indicates a lower quality of processing than ending the processing with a smaller block size 
S or finer grain detail. In one embodiment, the combination of block size and grain detail is 
utilized to determine a class and the class is utilized to determine a quality indicator value. 
In another embodiment, the combination of block size and data dependant matching error 
is utilized to determine a class and the class is utilized to determine a quality indicator 
value. The quality indicator value is then transmitted to the system controller 220 for use 

10 in asset allocation. 

Another quality indicator is described for noise reduction. Noise reduction entails 
analyzing picture content in classes, such as, for example flat unstructured regions (class 
1), edges and edge directions (class 2), and textured areas (class 3). The classes indicate a 
descending amount of impact on the visibility of noise. That is, flat unstructured regions 

15 (class 1) contribute the most to the visibility of noise while textured areas (class 3) 
contribute the least. In one embodiment, a noise level estimator can be utilized to 
distinguish between flie different classes^ 

A quality indicator value is determined based on the class processed with the 
available asset allocation. The higher the class processed, the larger the quality indicator 

20 value. Quality indicator values are then transmitted to the system controller 220 for use in 
asset allocation. In one embodiment, the system controller analyzes the received quality 
indicator values and the available assets and reallocates assets based on the information. In 
this embodiment, the system controller determines SVAs that will receive additional assets 
and SVAs that will receive less assets based on which asset distribution will maximize the 

25 overall output quality. 

In another embodiment, system controller 220 determines desired quality levels and 
transmits the quality levels to each SVA. In this embodiment, each S VA determines the 
amoimt of assets required to fulfill the quality level requirement and transmits the asset 
requirement to the system controller 220. In this embodiment, when system controller 220 

30 receives the asset requirements it can then optimize the system. 
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In an example, the system controller 220 optimizes the system 200 by determining 
the amount of miallocated assets and further determining the number of quality indicators 
for the SVAs that can be increased based on the amount of assets available. In this 
example, the system controller 220 optimizes the system 200 by increasing specific quality 
5 indicators or the amount of assets allocated to certain SVAs based on the determination. 
Alternatively, system controller 220 may optimize system 200 by decreasing specific 
quality indicators or the amount of assets allocated to certain SVAs based on the 
determination. 

FIG. 3 is a block diagram illustrating a scalable algorithm with quality indicator 
10 output in accordance with the present invention. In FIG. 3, scalable media algorithm 300 
includes scalable media processor 310 that is coupled to a quality control 320 and a quality 
indicator 330. In one embodiment and referring to FIG. 3, scalable media processor 310 
includes functions (31 1 - 314). 

Scalable media processor 3 10 receives input data, in the form of a signal, and 
1 5 processes the received data into output data, in the form of a signal, based on one or more 
control signals received from quality control 320. Scalable media processor 310 produces 
additional information that is transmitted to the quality indicator 330. 

In one embodiment, scalable media processor 310 is implemented as a scalable 
video algorithm (SVA). In other embodiments, scalable media processor 310 is 
20 implemented as a scalable graphics algorithm (SGA) or a scalable audio algorithm (SAA). 

In an example, scalable media processor 310 is implemented as a scalable video . 
algorithm (SVA) producing a quality indicator value for scalable motion estimation, 
described above. In another example, scalable media processor 310 is implemented as a 
scalable video algorithm (SVA) producing a quality indicator value for noise reduction, 
25 described above. 

Functions (31 1 - 314) conduct the actual processing of the input data scalable 
media processor 310 receives. Functions (311-314) may be implemented as scalable or 
non-scalable functions. In one embodiment and referring to FIG. 3, Functions 1 - 3 (31 1 - 
313) are scalable functions and receive control signals from quality control 320. In this 
30 embodiment. Functions 2 and 3 (3 1 2 and 313) provide information to quality control 320 
such as, for example class information, error information, and the like. The provided 
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infoimation enables quality control 320 to detennine a quality indicator value defining the 

scalable media processor 310 processing quality. 

FIG. 4 is a flow diagram depicting an exemplary embodiment of code on a 

computer readable medium in accordance with the present invention. FIG. 4 details an 
S embodiment of a method 400 for controlling asset allocation of a consumer terminal. 

Method 400 may utilize one or more systems detailed in FIGS- 1-3, above. 

Method 400 begins at block 410 where a system determines a requirement to 

control asset allocation. In one embodiment, the system is implemented as system 100 in 

FIG. 1 above. Method 400 then advances to block 420. 
10 At block 420, the system receives input data. The system may be implemented as a 

consumer terminal, a set-top box (STB), a TV set, a video display, and the like. In one 

embodiment and referring to FIG. 1 above, system 100 receives input data firom DVD unit 

1 10. Method 400 then advances to block 430. 

At block 430, the system processes the input data through Scaleable Media 
15 Algorithms (SMAs). In one embodiment, the scalable media algorithm is implemented as 

a scalable video algorithm (SVA). In other embodiments, the scalable media algorithm is 

implemented as a scalable graphics algorithm (SGA) or a scalable audio algorithm (SAA). 

In yet another embodiment, the processing is priority processing. 

In one example and referring to FIG. 1 above, the SVAs are implemented as any of 
20 the SVAs, such as, for example MPEG video decoder 130, sharpness enhancement 135,. or 

software Scaler 165. In another example and referring to FIGS. 2 and 3 above, the SVAs 

function substantially similar to SVA 210 or SMA 300. That is, the SVAs each receive an 

asset allocation, also referred to as a budget, from the system controller based on system 

requirements. Each SVA also receives input data and processes the received input data 
25 into output data based on the amount of assets allocated. Method 400 then advances to 

block 440. 

At block 440, the system determines quality indicator values associated with SVAs 
based on the processing. In one embodiment, the system determines quality indicator 
values associated with SVAs based on the amount of processing. In another embodiment, 
30 the system determines quality indicator values associated with SVAs based on the amount 
of processed data. In yet another embodiment, the system determines quality indicator 
values associated with SVAs based on the amoimt of processing and processed data. 



7 



wo 2004/075558 



PCT/IB2004/000241 



In one embodiment and referring to FIG. 2, quality indicator values are determined 
by a quality indicator unit that produces multiple indicator values based on specified 
criteria. In an example, the quality indicator for motion estimation includes specified 
5 criteria as detailed in FIG. 2 above. In another example, the quality indicator noise 
reduction includes ^ecified criteria as detailed in FIG. 2 above. The quality indicator 
values are then transmitted to the system controller. Method 400 then advances to optional 
block 450. 

At optional block 450, Hie system distributes assets based on the received quality 
10 indicator values. Block 450 is included to detail system functionality. In one embodiment, 
system assets are distributed as described in FIG. 2 above. Method 400 then advances to 
block 460 where it returns to monitoring the system for asset use changes. 

Alternatively, method 400 may continue to determine quality indicator values and 
reallocate assets. In one embodiment, this process includes processing the input signal 
15 through the scalable video algorithms based on the distributed assets, and determining at 
least one new quality indicator value associated with each scalable video algorithm based 
on the amount of processing and processed data for each scalable video algorithm. Assets 
can then be redistributed to each algorithm based on the new quality indicator values. 

In another embodiment, method 400 is implemented so that the system starts by 
20 determining quality levels for the SVAs. In this embodiment, the SVAs provide asset 
requirements and the system optimizes the resource usage based on the remaining assets 
after the asset allocation. 

In an example, method 400 provides at least one predetermined quality level for a 
plurality of scalable video algorithms and allocates assets to each scalable video algorithm 
25 based on the predetermined quality level. Additionally, the system can determine 

additional asset availability based on the allocation and reallocate assets based on the 
determination. In this embodiment, the predetermined quality level can be based on a user 
defined input. In an example, the user defined input is received via a user interface. 

In yet another embodiment, a regulator can be implemented to further control the 
30 asset allocation. In one embodiment, the regulator is implemented to control signal output 
quality. In this embodiment, the regulator controls quali^ to insure signal output quality 
remains within predetermined levels, on average, over time. 
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In another embodixnent, the regulator is implemented as a resource regulator. In 
this embodiment, the regulator controls processing to insure processing resources remain 
within predetermined levels, on average, over time. 

The above-described methods and implementation for controlling asset allocation 
of a consumer terminal are example methods and implementations. These methods and 
implementations illustrate one possible approach for controlling asset allocation of a 
consimier terminal. The actual implementation may vary from the method discussed. 
Moreover, various other improvements and modijBcations to this invention may occur to 
those skilled in the art, and those improvements and modifications will &11 within the 
scope of this invention as set forth in the claims below. 

The present invention may be embodied in other specific forms without departing 
from its essential characteristics. The described embodiments are to be considered in all 
respects only as illustrative and not restrictive. 
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